Descriptive Statistics

Analysis of Averages: I have calculated the mean of all the Spotify Audio feature’s for each of the playlists in my corpus. This will allow me to find the most typical and atypical tracks.

Danceability: Danceability seems to be higher in the playlists from the West Coast, highlighting the liveliness and positive mood of West Coast hip-hop music.

All the means are quite close to each other but the Southern playlist leans towards the West Coast in terms of danceability.

Valence: Valence is higher for the “classic” playlists, this trend is clear with the West Coast, East Coast and Southern Classic playlists. Songs during the classical period of Hip-Hop tended to be a lot more optimistic. Today’s rap is a lot more depressing.

Energy: Energy is highest in the Southern Classics playlist, this makes sense to me due to the very distinct production present in Southern American hip-hop.

Acousticness: Surprisingly enough acousticness seems to be stronger in the modern playlists. The weak nature or lack of acousticness in the old school playlists could be due to the novel presence of electronic music during the era of old school hip-hop. Tracks contained a lot more electric instruments such as electric synthesizers, guitars and drums. Modern hip-hop tends to be more acoustic with more use of instruments and human vocals.

Speechiness: The two East coast playlists are the most speechy which makes sense due the complex lyricism present in this style. Personally, West Coast songs generally have a more attractive beat and East Coast tracks usually have more interesting use of lyrics and wordplay.

Tempo: The modern playlists tend to have a higher tempo. This is because lyricism has become less important in hip-hop over the years.

Visualising the guesses

enables scrolling

These charts display four important metrics for each playlist in my corpus. We compare valence, energy, loudness and mode in this comprehensive visualization. We can clearly see that the scatter plots of the classical playlists resemble each other much more than the modern ones. The modern playlist tracks plots are more dense than the classical ones, representing less variety and range musically. The tracks of the classical playlists differ more each other. This could be due to the fact that hip-hop was an emerging genre without fixed parameters or guidelines. We can conclude that time-based differences are easier to identify than coastal styles in this genre.

Most average songs

In the upcoming sections some track level analysis is conducted. To have some more or less scientific grounds of choosing a representative track, I created a function that chooses quite literally the most average track in the playlist (compared to other tracks in the same playlist) by Spotify’s metrics.

The function calculates the sum of absolute distances between each feature of the songs in the playlist and the feature’s mean. The songs with the minimum difference are chosen to be the ‘most average’.

Here’s the function’s code:

get_most_average_song <- function(playlist) 
{
  metrics<-c("danceability","energy","key","loudness","mode","speechiness",
             "acousticness","instrumentalness","liveness","valence","tempo")
  
  mean_metrics=colMeans(playlist[,metrics],na.rm = TRUE)
  playlist$diff=rep(0,nrow(playlist))
  
 for(i in 1:nrow(playlist)) {
  playlist[i,'diff']<-sum(abs(playlist[i,metrics]-mean_metrics[metrics]))
 }
  song=playlist$track.name[which.min(playlist$diff)]
  return (list(playlist,song))
}

The most average song from I love My West Coast Classics is Fuck Tha Police.

The most average song from I Love My East Coast Classics is Loungin (Who Do Ya Luv) - Remix.

The most average song from Cali Fire is Paranoia / #1 Stunna.

The most average song from State of Mind is GET MONEY (feat. DDG & Tyla Yaweh).

Pitch Analysis Classics

The first thing to point out is that Loungin (East Coast) is a lot shorter. Its pitch magnitudes lies strongest at A#/Bflat, A, F#/Gflat. Fuck Tha Police contains stronger magnitude in the keys of C, C#/D flat as well as A and B.

We can conclude that early hiphop songs are dominant in the keys of A.

Pitch Analysis Modern

Here, again, the East Coast song “GET MONEY” is a lot shorter, pitch mangitude is strongest at C#/Dflat, D#/Eflat and G#/Aflat. Paranoia (West coast) reveals very strong mangitude in the keys of C, C#/D flat, E and B

We can conclude that modern hiphop songs of these coasts are dominant in the keys of C, C#/D and B.

Modern hiphop is much more diverse in its pitches, this again can be linked to the advanced productions present in the genre nowadays and can also point to Spotify algorithm being able to detect it better.

Chord and Key Comparison of two old school hip-hop songs

Fuck Tha Police

Loungin (Who Do Ya Luv) - Remix

enables scrolling

The Chords look fairly similar between coasts. The keys are very different. The East coast song is mainly in minor keys, except for one moment around 90 seconds. The West coast song has more diversity in keys.

Chord and Key Comparison Modern

Paranoia / #1 Stunna

GET MONEY (feat. DDG & Tyla Yaweh)

enables scrolling

The Modern songs are more different between coasts.The West coast song has two periods where there’s more intensity in both Chords and Keys, while for the East coast song things only happen in the very beginning.

Tempo Novelty

Classics

Modern

enables scrolling

These plots potentially suggest a great feature to differentiate between the old and new songs-tempo novelty in the beginning of the song. Both old songs start with a big drop, while the new songs do not have it. Also, East coast songs seem to change tempo more frequently compared to the West ones.

Classification

Call:
 randomForest(formula = playlist ~ ., data = training_set, ntree = 1000,      mtry = 15, importance = TRUE, confusion = TRUE) 
               Type of random forest: classification
                     Number of trees: 1000
No. of variables tried at each split: 15

        OOB estimate of  error rate: 40.5%
Confusion matrix:
                    East Coast Classics East Coast Modern West Coast Classics
East Coast Classics                  29                 2                  16
East Coast Modern                     4                33                   1
West Coast Classics                  11                 4                  32
West Coast Modern                     4                15                   6
                    West Coast Modern class.error
East Coast Classics                 3        0.42
East Coast Modern                  12        0.34
West Coast Classics                 3        0.36
West Coast Modern                  25        0.50

Here a random forest model is estimated. The model was tuned to 1000 samples and the tree depth of 15. This seems to work well as we have quite a few features to choose from at each step (tree depth), but also our training set is relatively small (4x50=200), especially taking into account that we have 4 classes to choose from. Hence, averaging 1000 observations is reasonable.

Classification error may look high (>30%) at the first glance. However, we are dealing with music and some mistakes are more forgivable than others: although, the algorithm could be better in distinguishing between East and West, it distiguishes very well between time epochs! Classics are almost always classified as Classics, while Modern is almost always Modern.

It is also interesting that precision is significantly higher for some playlists than for others. The confusion matrix suggests that East Coast Modern and West Coast Classics are the most different from the rest of the playlists (classified correctly most often). West Coast Modern is the toughest one to crack - half of the songs are indeed their own genre, but some other 15 songs resemble East Coast Modern, whose features we seem to have identified quite well, and of course some inspiration from West Coast Classics!

Fun Fact: West Coast Modern a.k.a. the “Cali Fire” playlist is curated by Apple Music, not Spotify. This may become a strating point in uncovering the differences in algorithms between the two companies.

All in all, the model seems very reasonable given a relatively small training set, 4 classes, and qualitative nature of music itself.

The feature importance graph (next tab) shows that tempo, speechiness, instrumentalness and c05 are important in differentiating among playlists. This, on the one hand, is helpful in playlist comparison, but also to suggest which features are the most important to look at when aiming to classify new songs.

Feature importance

Revisiting

Now that we know which features are the most impactful, we can revisit my first plot with the new features on the axes. Immediately, we see the similarities within Classics (more clustered in the bottom left corner) and Modern (more spread). In addition, the diffrence in instrumentalness (size on the plot) is also very persistent. This plot also illustrates why it is hard for the model to classify West Coast Modern.

What about South?

There’s a long standing debate about where Southern Rappers drew inspiration from: East or West? Modern or Old School? I already looked at Southern Rap in the mean analysis. Now, as a fun exercise, I will use my random forest model trained on the other 4 playlists of my corpus to classify the songs from the South playlist. But before that, let’s look at the most important features in the context of the South.

From a preliminary glance, I suspect classification to be a difficult task. The Speechiness-Tempo (x-y) positions of the songs resemble more the Modern playlists, with a few very instrumental songs as in the Classics.

The moment of truth! Predicting the South

predictions
East Coast Classics   East Coast Modern West Coast Classics   West Coast Modern 
                  2                  27                   9                  12 

The moment of truth! The predictions indeed indicate that the Southern Rap is more similar to Modern styles (mostly East, but that’s also our best defined class), with a bit of the West Coast Classic sound mixed in (probably the instrumental ones). Also, we can be pretty confident that Southern Rap is almost nothing like East Coast Classics.

These are some exciting findings for any rap or music lover, as they computationally quantify such unquantifiable concepts as influence and inspiration.

P.S. The most average song from I Love My Down South Classics is Get Throwed.

Bonus Individual track predictions

I hope this works